Learning Novel Concepts in the Kinship Domain

This paper addresses the role that novel concepts play in learning good theories. To concretize the discussion, I use Hinton’s kinship dataset as motivation throughout the paper. The standpoint taken in this paper is that the most compact theory that describes a set of examples is the preferred theory—an explicit Occam’s Razor. The kinship dataset is a good test-bed for thinking about relational concept learning because it contains interesting patterns that will undoubtedly be part of a compact theory describing the examples. To begin with, I describe a very simple computational level theory for inductive theory learning in first-order logic that precisely states that the most compact theory is preferred. In addition, I illustrate the obvious result that predicate invention is a necessary part of any system striving for compact theories. I present derivations within the Inductive Logic Programming (ILP) framework that show how the intuitive theories of family trees can be learned. These results suggest that encoding regular equivalence directly into the training sets of ILP systems can improve learning performance. To investigate theories resulting from optimization, I devise an algorithm that works with a very strict language bias allowing all consistent rules to be entertained and explicitly optimized over for small datasets. The algorithm, which can be viewed as a special case implementation of ILP, is capable of learning a theory of kinship comparable in compactness to the intuitive theories humans use regularly. However, this alternative approach falls short as it is incapable of inventing the unary predicate sex to learn a more compact theory. Finally, I comment on the philosophical position of extreme nativism in light of the ability of these systems to invent primitive concepts not present in the training data. Introduction eral because of the semi-decidability of first order logic, there has been great success at the algorithmic level in the field of Inductive Logic Programming (ILP). The problem ILP adThe core of the intuitive theory of kinship in western culture dresses is: learn a first-order logic theory that, together with is the family tree, from which any number of queries about provided background knowledge, logically entails a set of exkinship relationships can be answered. Could a machine, preamples (Nienhuys-Cheng and de Wolf, 1997). sented with the kinship relationships between individuals in a family, learn the intuitive family tree representation? Using the ILP framework, it is possible to show how inverse resolution can devise all three of the basis set predicates that This paper focuses heavily on a dataset introduced in Hinton comprise the family tree representation. The most interesting (1986). In this dataset, a group of individuals are related by result is the discovery of sex which requires that logical enthe following relations: father, mother, husband, wife, son, codings of regular equivalence classes can be combined in an daughter, brother, sister, uncle, aunt, nephew, niece. The inverse resolution step to generate the new predicate. This family tree representation efficiently encodes all of these rela­ result suggests that explicitly encoding regular equivalence tionships using a basis set composed of spousal relationships, parent/child relationships and the sex attribute. To learn and other second-order properties of relational datasets may contribute to their learnability. this theory, a machine would have to first invent the basis set and then redefine the existing relations in terms of this basis To investigate the computational level, I devise a special-case set. How could a machine discover such a basis set? version of ILP that is optimized to use a very strict set of According to my computational level theory, the basis set is restrictions on the type of theories it can entertain. By tradnot discovered at all. Rather, it is a byproduct of an optiing expressibility for tractability, it is possible to explicitly mization process that searches for the most compact theory optimize over the set of all possible rules for each relation inthat entails a set of examples. At the algorithmic level, the dividually. Unfortunately, optimizing across the relations is basis set could possibly be discovered through the process of intractable. The resulting rules can be further compressed by local optimizations that lead to more compact theories. using inverse resolution to invent new predicates that simplify existing ones. The resulting theory for the kinship domain is While the computational approach is not computable in gencomparable in compactness to the family tree representation. The sex of the individual is often implicitly specified by the gender of the name. Personal communication and class notes of J. Tenenbaum (Tenenbaum, 2004) The regular equivalence classes for the kinship dataset are all pairs of generations and sex in the family tree White and Reitz (1983); Kemp et al. (2004).